Phoenix and Plug benchmark improvs #32
Conversation
fix for phoenix
|
I'm also adding some performance improvements for plug and phoenix on this PR soon. |
|
Done! Now Plug and Phoenix are in prod env. ;) |
|
Yeah, running Plug in prod instead of dev is definitely smarter. I did tests with a couple with Plug in prod instead of dev and got this: Language (Runtime) Framework (Middleware) Max [sec] Min [sec] Ave [sec]
------------------------- ------------------------- --------------- --------------- ---------------
rust iron 4.559537 4.498832 4.534531
elixir plug 5.444713 5.185388 5.326322
crystal router_cr 9.139290 8.827565 9.007032
node express 32.212435 31.668852 31.824345I'm not sure why router_cr on the chart is showing to be so much faster than rust, I could not replicate that at all. Also the Elixir/Plug one might be a touch bit faster if it is warmed up first, but that might already be done so if so ignore this sentence. :-) EDIT: For note, my tests above were done on a very very old AMD Phenom ][ at 3.6ghz 6 native cores with 16 gigs ram freshly rebooted with Ubuntu 17.04 and entirely unused (not even X GUI running) |
|
Added Phoenix latest pull above: Language (Runtime) Framework (Middleware) Max [sec] Min [sec] Ave [sec]
------------------------- ------------------------- --------------- --------------- ---------------
rust iron 4.559537 4.498832 4.534531
elixir plug 5.444713 5.185388 5.326322
elixir phoenix 5.792088 5.673842 5.752746
crystal router_cr 9.139290 8.827565 9.007032
node express 32.212435 31.668852 31.824345 |
|
Don't forget to update the README.md as well as it states to do for PR's. |
|
@OvermindDL1, the README is already with Phoenix, it was added by another PR, this one is just a performance improvement. |
|
I ran some more, still sorted from fastest to slowest as top to bottom: Language (Runtime) Framework (Middleware) Max [sec] Min [sec] Ave [sec]
------------------------- ------------------------- --------------- --------------- ---------------
rust iron 4.559537 4.498832 4.534531
rust nickel 5.253805 5.165856 5.207764
elixir plug 5.444713 5.185388 5.326322
elixir phoenix 5.792088 5.673842 5.752746
rust rocket 5.977488 5.849199 5.914374
crystal router_cr 9.139290 8.827565 9.007032
crystal kemal 10.316980 9.544119 10.050472
node express 32.212435 31.668852 31.824345
ruby roda 66.817984 63.877961 65.617340
ruby sinatra 147.016259 144.732534 146.124146
ruby rails 477.844824 475.863197 476.804092I could not ╰─➤ make go
go get -u github.com/labstack/echo
package context: unrecognized import path "context" (import path does not begin with hostname)
Makefile:60: recipe for target 'echo' failed
make: *** [echo] Error 1I could not ╰─➤ make swift 1 ↵ cd swift/vapor; swift build --configuration release
error: unsatisfiable
Makefile:103: recipe for target 'vapor' failed
make: *** [vapor] Error 1I'm getting some very different results from the chart on the repo, I'm wondering if I have a higher amount of (though slower) cores thus the concurrent ones work better? |
|
And for note: ╰─➤ go version
go version go1.6.2 linux/amd64╰─➤ swift --version
Swift version 3.1.1 (swift-3.1.1-RELEASE)
Target: x86_64-unknown-linux-gnu |
|
I made a PHP file of (this is to follow the requirements in the README.md precisely): <?php
$uri = $_SERVER['REQUEST_URI'];
if($uri == "/") die("");
else if($uri == "/user") die("");
else if(substr($uri, 0, 6) == "/user/") die(substr($uri, 6));
?>And tested with that using the hhvm php server since that is the only one I have installed at the moment, added the results to the rest here: I'm impressed, PHP is pretty dang fast, especially considering it has to hit the filesystem to check if the file's been changed on every request. PHP utterly blows Ruby away in speed, but still does not beat crystal/elixir/rust... |
|
@kelvinst @OvermindDL1 Or which version of crystal and llvm did you use? |
|
@OvermindDL1 PHP vs ruby is pretty unfair. Don't forget roda and sinatra are still pretty large libraries in contrast to 4 line PHP file :) Can you try https://github.com/codeguy/Slim (found via google "PHP sinatra like frameworks") or something similar. |
|
Thanks!!! |
From the makefile yes.
╰─➤ crystal --version
Crystal 0.22.0 [3c71228] (2017-04-20) LLVM 3.5.0It should be the latest in the repositories from the offical site? I don't really use crystal so I just rely on it being updated that way. :-)
Hence why I did not turn it in to a PR, it was just for my own morbid curiosity, and since the readme says 'as fast as possible'... ^.^ If you want I can set up a 'proper' test with that slim framework and PR it in, or just post the results here if you are curious? |
|
I got a few minutes free here. @schovi or anytone else if curious, I changed my basic PHP test to use slim instead (holy hell that thing is huge) and got these results (ordered from fastest to slowest, top to bottom again): Language (Runtime) Framework (Middleware) Max [sec] Min [sec] Ave [sec]
------------------------- ------------------------- --------------- --------------- ---------------
rust iron 4.648752 4.546433 4.597737
elixir plug 5.498771 5.376086 5.452387
rust nickel 5.570673 5.506968 5.541427
elixir phoenix 6.101516 5.942020 6.042626
rust rocket 6.786611 6.330680 6.452137
crystal router_cr 8.769548 8.018409 8.528628
crystal kemal 12.281298 10.394767 11.516996
php slim 76.066147 74.935423 75.512605The php file is: <?php
require 'vendor/autoload.php';
$app = new Slim\App();
$app->get('/', function ($request, $response, $args) {
return $response->write("");
});
$app->post('/user', function ($request, $response, $args) {
return $response->write("");
});
$app->get('/user/{id}', function ($request, $response, $args) {
return $response->write($args['id']);
});
$app->run();
?>Definitely slowed down a lot. 😆 Elixir Phoenix is definitely the full-stack Rails-like contraption. Elixir Plug is the base-most simple router and such helpers thing. Elixir holds its own very well. :-) And I confirmed that crystal was compiled with |
|
@OvermindDL1 Anyway, such a feedbacks are very welcome!!:smile: |
It is fun! Plus I am extremely curious why the crystal times were so different for me than on the chart, it makes me think I'm doing something wrong, but don't know what... :-( I'm wondering if it is just because my system has more physical cores, which the rust and elixir versions can take advantage of better. Maybe I should run them restricted to 2 cores, hmm... |
|
And here are the results with crystal, rust, and elixir with them constrained to only the first 2 out of the 6 physical cores, in top-to-bottom as fastest-to-slowest order as usual: Language (Runtime) Framework (Middleware) Max [sec] Min [sec] Ave [sec]
------------------------- ------------------------- --------------- --------------- ---------------
crystal router_cr 5.083894 4.410376 4.807896
rust nickel 5.622460 5.601752 5.616635
crystal kemal 7.244543 6.778164 6.980066
rust rocket 7.212035 6.947049 7.023690
rust iron 7.732281 7.595761 7.671830
elixir plug 12.045980 11.741010 11.872830
elixir phoenix 13.847535 13.547675 13.637995Running them again while allowing them to use all 6 cores returns them to their prior speeds and orders with crystal being the slowest again, which is slower than the 2-core test, so I'm thinking that crystal is not multi-threaded well because I know that both Rust and the EVM (that Elixir runs on) are both able to be threaded crazy-well (with the EVM being almost linearly parallizable so it should eventually win above all with 'enough' cores). Hmm, wish I could test on my 24 core server right quick, unable to any time soon though... But yeah, testing on a comparatively tiny Mac computer (which really, no one would ever host servers on) is definitely not where the tests should be run. ^.^ |
That would be reeeeeally interesting!!!!!!! 😮 |
fix for phoenix